Elena
Sun Nov 17 2024
|
6 answers
1575
What is a block manager in Spark?
The BlockManager in Spark is a key-value distributed storage system tailored for Spark. It runs as a local cache on all nodes, including drivers and executors, providing consistent get and set interfaces for data blocks stored in memory, disk, or off-heap. The BlockManagerMaster manages all BlockManagers in the cluster, coordinating data block replication and migration.
